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I. Introduction 


Since Coleman et al. (1966), researchers have sought to understand the 
relationship between school expenditures and educational outcomes. Early studies 
cast doubt on the idea that school spending is strongly linked with student 
performance (Clark, 2003; Hanushek, 2003; Papke, 2008; Roy, 2011). More 
recently, however, researchers have used causal identification strategies to provide 
evidence that overall spending plays a more influential role than previously thought 
(Jackson et al., 2016; Candelaria & Shores, 2017; Hyman, 2017; Miller, 2017; 
Gigliotti & Sorensen, 2018; Lee & Polachek, 2018; LaFortune et al., 2018; Jackson 
et al., 2020). The extent to which school expenditures impact student outcomes is 
of particular interest today as the COVID-19 pandemic creates significant state 
budget shortfalls while simultaneously increasing fiscal demands on schools. 

Evidence from previous recessions may provide pertinent information that can 
be used to ease the impact of the COVID-19 pandemic on schools and students. 
Prior studies have claimed to uncover significant causal relationships between 
recessionary changes and important educational factors such as teacher quality and 
retention (Nagler, Piopiunik, & West, 2017; Fuchsman & Zamarro, 2019). The 
focus of this paper is one such study, a forthcoming American Economic Journal: 


Economic Policy article titled “Do School Spending Cuts Matter? Evidence from 


The Great Recession,” that investigates the impact of spending cuts related to the 


Great Recession on test scores and college-going rates (Jackson, Wigger, and 
Xiong, 2020a; JWX hereafter). The paper exploits state-level variation in education 
spending using an instrumental variables (IV) approach to identify the causal effect 
of recessionary spending cuts on student outcomes. 

JWX uses pre-recession (i.e., 2008) K-12 state revenue share to instrument for 


the effect of expenditures on educational outcomes. This approach relies on the 


assumption that states which depend more heavily on state-generated education 
funding are more vulnerable to recessionary changes. The authors argue state-share 
captures recession vulnerability because it is subject to crowd-out from other more- 
pressing budgetary demands, such as Medicare and unemployment benefits, and 
because state income and sales tax revenues are more susceptible to the business 
cycle than local revenue, namely property taxes. 

We replicate JWX as a first step in modeling the potential effects of the pandemic 
on K-12 spending and outcomes. This paper describes our process of duplicating 
JWX and highlights a variety of tests we employ to investigate the nature and 
robustness of the relationship between school spending reductions and student 
outcomes. Specifically, we explore how the findings differ when we implement 
different modeling choices around state groupings and year-specific controls. We 
also investigate the extent to which K-12 spending may be confounded with higher 
education spending and the implications this may have on capturing changes in 
college enrollment. 

While we are able to reproduce the JWX findings, we do not find consistent 
evidence that spending cuts resulting from the Great Recession had a causal effect 
on student outcomes. Instead, we find: 

e the JWX group IV model categorizes very few states as having low- 
reliance (three states) or high-reliance (four states) on state revenue to 
fund K-12 education, and while there may be an effect at the extremes 
of the distribution, the results are sensitive to relatively small changes 
in group composition; 

e the JWX linear IV uses a non-standard specification for year fixed 
effects, and the results are sensitive to using other reasonable 
specifications to control for year-specific shocks; and 

e post-recession K-12 and higher education funding trends are related, 


and as a result, the JWX finding that K-12 spending reductions 


impacted state college-going rates is likely confounded with 
contemporaneous higher education funding trends. 

The rest of the paper proceeds as follows. First, we briefly describe the JWX 
study and findings. Section II describes the data used to reproduce JWX. Section 
I motivates our robustness checks. Our analytical approach is described in Section 
IV. Results are outlined in Section V and Section VI concludes the paper, providing 


implications of our findings. 


Description of “Do School Spending Cuts Matter? 


Evidence from The Great Recession” 


JWX uses an instrumental variables (IV) analysis to causally link spending 
reductions resulting from the Great Recession to student outcomes. Since 
unobserved factors influence both per-pupil expenditures (PPE) and student 
outcomes, PPE is considered endogenous. Therefore, researchers cannot directly 
investigate the causal impact of PPE on student outcomes. Isolating the effect of 
changes in PPE requires finding an instrument that is causally linked to PPE, has 
no impact on the outcome measure except through PPE, and is not caused by any 
factors that also impact student outcomes. 

JWX exploits variation in state-appropriations to public education as an 
instrument for per-pupil spending. Specifically, the authors utilize the share of 
public-school revenues generated from state sources just prior to the Great 
Recession in place of per-pupil spending (i.e., 2008 state share of K-12 revenue). 
This identification strategy relies on the fact that state-level taxes are more 
susceptible to the business cycle than either local property taxes or federal funding. 
Therefore, states which are more reliant on state funding are likely to experience 


larger recession-induced cuts to school spending. 


JWX presents evidence that spending reductions cause declines in both student 
achievement and college matriculation rates. The authors find that for every $1,000 
cut in per-pupil spending attributed to the recession, scores on the National 
Assessment of Educational Progress (NAEP) exam decline by 3.85 percent of a 
standard deviation. Results are similar for college-going rates, where a $1,000 
reduction in per-pupil expenditures results in a decrease in college enrollment of 
1.24 percentage-points. In addition to the main analysis, JWX finds that spending 
reductions also increased achievement gaps by both race and socioeconomic status, 
and that states that cut K-12 spending hired fewer personnel and reduced capital 
expenditures rather than core K-12 expense categories. 

The JWX findings suggest that both student achievement and attainment may be 
broadly responsive to fluctuations in education spending. Moreover, their results 
imply that high-poverty districts — where state aid makes up a larger proportion of 
revenues — may be the most at-risk to changes in school funding. Naturally, these 
findings have important implications for policymakers interested in understanding 


the relationship between education finance and student outcomes. 
II. Data 


Our dataset, which mirrors the sources and strategies employed in JWX, contains 
data on all 50 states and the District of Columbia spanning the years between 2001 
to 2019. We closely match the mean, standard deviation, and number of 
observations for all data elements used in JWX’s main analyses. Table 1 provides 
summary statistics for our data and compares them to the JWX data.! 

Unadjusted test scores for the National Assessment of Educational Progress 


(NAEP) were obtained from publicly-available data aggregated and hosted by the 


' See JWX Table 1 for summary statistics. 


Urban Institute.? To match the JWX data we use public school state-year average 
scores for 4" and 8" grade math and reading assessments between 2002 and 2017. 
All scores are standardized to a base year of 2003 using the national public school 
NAEP test means and standard deviations." 

College-going data come from the Integrated Postsecondary Education Data 
System (IPEDS). The data include information reported by institutions on the 
number of first-time college freshmen who graduated from high school in the past 
12 months, aggregated by state of origin.* This portion of the survey is only 
administered during even years. To compute a college-going rate, we obtain 
population estimates from the US Census for the number of 17-year-olds in each 
state the year prior to enrollment.° Our college-going rate divides the number of 
enrollees from each state by the number of 17-year-olds in the state the previous 
year.° This rate is then associated with the year which matches the 17-year-old 


observation and NAEP administration years. 


High school graduation rates are obtained from the National Center for Education 


Statistics (NCES). NCES changed its high school graduation rate measure in 2014. 
As a result, data from 2001-2013 represent Average Freshman Graduation Rate 
(AFGR) values and data from 2014-2019 are Adjusted Cohort Graduation Rates 
(ACGR).’ 


? Five states do not have 2002 NAEP scores: AK, CO, NH, NJ, and SD. To generate a balanced panel of data, we impute 
the mean reading z-score for those same states in 2003. Results are consistent both including and excluding these 
observations. 

3Data obtained from the NAEP Data Tool (https://www.nationsreportcard.gov/ndecore/landing). 2003 National NAEP 


Exam Statistics: 4th grade reading —- Mean = 216, SD = 37; 8th grade reading — Mean = 261, SD = 35; 4th grade mathematics 
— Mean = 234, SD = 28; 8th grade mathematics — Mean = 276, SD = 36 


+ We use files labeled “Residence and migration of first-time freshmen: Fall 2018” and use the associated downloadable 
STATA .do file provided by IPEDS to organize and summarize the data. 

> Data from 2000-2010 come from the State Intercensel Estimates (https://www.census.gov/data/datasets/time- 
series/demo/popest/intercensal-2000-2010-state.html) and data from 2010-2019 come from the Vintage 2019 Estimates 
(https://www.census.gov/data/tables/time-series/demo/popest/2010s-state-detail.html). 

6 JWX reports using the average number of 17- and 18-year-olds in their Data Appendix. There was some ambiguity 
around these data in previous versions of the paper. We use the number of 17-year-olds, which yields a rate that is not 
statistically different from their measure. 

7 NCES reports a summary of the difference here: https://nces.ed.gov/blogs/nces/post/what-is-the-difference-between- 


the-acgr-and-the-afer. 


We obtain school finance data from the Census (F33) School District Finance 
Survey available at the Common Core of Data (CCD) . The data contain 
information on school revenue and expenditures, as well as the number of students 
enrolled in each district and state. To obtain per-pupil expenditures, spending 
variables were CPI-adjusted to 2015 dollars and divided by total district 
membership in each year.® 

We also retrieve data on economic conditions in each state and year from the 
Bureau of Labor Statistics (BLS). Following JWX, we use these data to construct 
Bartik economic control variables for unemployment and average wage. This 
process is described in A.2 of the online appendix for JWX. We obtain state 
employment shares for 2007 — the first year of the recession — using BLS “CSVs 
By Industry” and “Annual Averages.”?!° Average wage by industry is obtained 
from BLS Occupational Employment Statistics data. Unemployment data by 
industry and year are obtained from the BLS Economic News Release.!!!” 

Information on higher education finance is obtained from two sources. State share 
of higher education funding comes from the IPEDS finance surveys. Data on state 
and local appropriations per FTE and tuition and fees per FTE are obtained from 


the State Higher Education Executive Officers Association (SHEEO, 2019). 


8 JWX winsorizes extreme values for per-pupil expenditures by capping values of districts with expenditures great that 
200 percent the 99" percentile of per-pupil revenues or less than 50 percent of the first percentile. Our analysis shows that 
the mean and standard deviation of spending data are not significantly changed when winsorizing, therefore we utilize the 
non-winsorized data in our study. Our non-winsorized spending data is not statistically different from JWX winsorized data. 

* https://www.bls.gov/cew/downloadable-data-files.htm 

'0 Census industry code crosswalk used for the BLS unemployment rate can be found here: 
https://www.census.gov/topics/employment/industry-occupation/guidance/code-lists.html 

" For example, data for 2019 and 2020 can be found at: https://www.bls.gov/news.release/empsit.t14.htm. 

"2 While JWX does not provide any summary statistics for these Bartik control variables, the 2019 NBER working paper 
version (Jackson et al., 2019) does provide Figure A4 depicting yearly averages for the unemployment rate Bartik. The figure 
is not well labeled but our values generally match this figure. Our average Bartik control for the unemployment rate is: 2001- 
4.74%, 2002-5.82%, 2003-5.97%, 2005-4.99%, 2007-4.49%, 2009-9.15%, 2011-8.56%, 2013-6.94%, 2015-4.97%, 2017- 
4.09%. 


III. Motivation for Robustness Checks 


Econometric models can be constructed in a variety of ways, and many modeling 
choices may be somewhat arbitrary or theoretically unimportant. However, if the 
model’s estimates represent the true causal impact, they should be consistent across 
many different reasonable ways of constructing the model. JWX presents several 
robustness checks for their analysis; however, there are a few sensitivity tests they 
do not employ. As part of our replication we test the sensitivity of the analysis to 
various additional modeling choices we believe are important to assess. The 
remainder of this section describes the theoretical motivation that underlies our 
approach and describes the specific tests we use to investigate the nature and 
robustness of the relationship between school spending cuts due to the Great 


Recession and student outcomes. 


K-12 Spending and Outcomes Trends 


To motivate their analysis, JWX compares trends over time for PPE and student 
outcomes, demonstrating that they seemingly move together. We follow suit, 
graphing trends in PPE, NAEP scores, and the college-going rate. Figures 1-3 
depict these trends. Our figures show that PPE declined during the Great Recession 
but increased sharply after 2013. We also include NCES’s projections for 2018 and 
2019 (dashed blue lines), which show that PPE has likely continued to climb in the 
years after the JWX study period. 

As indicated in our graphs, both NAEP scores and the college-going rate 
increased with PPE during the years leading up to the recession. However, it is less 
apparent whether PPE and student outcomes have been moving together after the 
outset of the recession. Figure 1 shows that while PPE rebounded a few years after 
the recession, NAEP scores have continued to decline. The national college-going 


rate for 17-year-olds, depicted in Figure 2, leveled off following the recession and 


has remained flat even as PPE increases. The recent divergence between PPE and 
student outcomes, especially NAEP scores, raises the question of whether the 
seeming correlation between the variables in earlier years was simply a 
coincidence. 

The JWX identification strategy relies on differences between states in post- 
recession spending. To illustrate how state-level K-12 expenditures changed 
following the Great Recession, Table 2 provides constant dollar PPE differences 
over three periods: 2007-10 (3-year), 2007-12 (5-year), and 2007-2017 (10-year). 
While average PPE decreased following the great recession, not all states 
experienced declines, and several saw sizeable increases. Only a little over half of 
states (i.e., 26) states saw spending reductions between 2007 and 2012. Nine states 
experienced spending increases greater than $1,000 per student over this same 
period. 

Changes in spending appear to only be weakly correlated with JWX’s measure 
of pre-recession state K-12 revenue share. The 3-year spending changes have a 0.28 
correlation with state share while the 5-year and 10-year changes have correlations 
of 0.10 and 0.23, respectively. Sections [V and V more rigorously investigate the 


relationship between PPE and student outcomes. 
Instrument Data 


Both the U.S. Census Bureau and NCES provide data on K-12 revenue by source. 
JWX uses Census data to define their instrument.'? These two institutions each 
make their own unique determinations regarding what is state versus local revenue, 


leading to significant differences in revenues attributed to each category for some 


'3 The JWX state revenue share data can be found in Table A2 of the online appendix for the forthcoming version (Jackson 
et al., 2020a). The 2008 Census state share data can be downloaded at 

https://www.census.gov/data/tables/2008/econ/school-finances/secondary-education-finance.html. There are some 
unexplained differences between JWX’s data and Census that may stem from Census data updates. 


states. For example, Census lists Arkansas’ state revenue share as 76 percent, 
counting the state mandated minimum local property tax effort as state revenue (see 


note on page B-1 of the Census 2008 survey documentation). On the other hand, 


NCES data for 2008 show Arkansas’ state revenue share at 56.7 percent, nearly 20 


percentage points lower than Census. Given the purpose of the instrument is to 
capture recession vulnerability, the NCES data for state share is likely, at least for 
Arkansas, a better fit for the analysis. 

Table 3 summarizes the 2008 state revenue share data from JWX (Column 1), 
Census (Column 2), and NCES (Column 3).'* In total, there are 13 states where 
JWX and NCES data for 2008 state revenue share differ by more than two 
percentage points. Six states differ by more than five percentage points and three 
states vary by more than 15 percentage points.'° These differences are potentially 
important given how few observations are included in the analysis (1.e., the 50 states 
and Washington D.C.). 

For some states, the official data sources do a poor job capturing the unique nature 
of their school funding systems. In Vermont, for example, public education is 
largely funded through a dedicated statewide property tax.!° In total, between 60 
and 70 percent of Vermont’s K-12 education funding comes from property taxes.” 
Because the property tax is statewide and distribution is managed through the state, 
both Census and NCES data list Vermont’s state share as between 85 percent and 
90 percent. JWX lists Vermont as having 68 percent state revue share. The state 
share instrument is intended to measure reliance on types of taxes and an allocation 


process that would make education funding more susceptible to cuts during a 


4 NCES also collects information property tax revenue, which is a plausible substitute for the JWX instrument. The 2005- 
07 state-level property tax share calculated using NCES data (Table 2, column 5) has a 0.65 correlation with the state revenue 
share documented in table A2 of JWX (Table 2, column 1). 

'S Tn addition to how state mandated local tax effort is categorized, Census and NCES also handle pension payments 
somewhat differently. 

'© A description of Vermont’s school funding system is available at EdBuild. http://funded.edbuild.org/state/VT 

'T https://ljfo.vermont.gov/assets/Uploads/d76744a36e/The-Education-Fund-and-Education-Finance-v2.pdf 
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recession. However, that is far from the case for Vermont, which relies in large part 
on dedicated, stable property taxes. Vermont increased its spending per pupil by 
$1,919 in the years immediately following the Great Recession (i.e., 2007-12), the 
second largest increase over this period (Table 2 column 3). 

JWX does not explain why state-mandated/collected property tax revenue is 
handled differently for California than it is for Arkansas and Vermont. California, 
like the other two states, has a state mandated local property tax effort. JWX argues 
that California’s property tax dollars should be categorized as local revenue because 
they are not subject to crowd-out and are more stable than other state-level taxes. 
However, the same is true in Vermont and Arkansas, whose property taxes are 
counted as state revenue in JWX. 

Washington D.C. also presents a challenge when categorizing revenue by source. 
Like other states, education is primarily financed through a combination of local 
property taxes, general fund appropriations, and federal funds. However, the D.C. 
government combines state and local functions under one umbrella.'® Since it is not 
technically a state, neither NCES nor Census categorize D.C.’s general fund 
appropriations as state revenue. NCES simply lists Washington D.C.’s state 
revenue as “not applicable,” while Census reports leave state revenue blank. JWX 
lists Washington D.C.’s state revenue share as zero. Given the theoretical 
justification for the instrument and D.C.’s reliance on general fund appropriations 
for a large share of school funding, a value greater than zero is likely more 
appropriate. In fact, since the entire school budget is appropriated by the council, 
there is a reasonable argument that D.C.’s state share should be 100 percent. 


Because of this uncertainty around how to handle Washington D.C.’s state K-12 


'8 A description of Washington D.C.’s school funding system is available at NCES and EdBuild, both linked below. 
https://nces.ed.gov/edfin/pdf/StFinance/District.pdf 
http://funded.edbuild.org/state/DC 
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revenue share, we drop the jurisdiction from analyses that use this instrument 
except when we are explicitly replicating JWX.!” 

The last two columns of Table 2 present our measures of pre-recession state and 
property tax K-12 revenue share. While JWX uses 2008 as its pre-recession year 
for the instrument, we want to test measures that more clearly capture the period 
before the recession began and that smooth out potential year-to-year variation. To 
that end, both our state and property tax share measures are averaged over the years 
2005 to 2007. We include property tax share as an alternative instrument that fulfills 
the theoretical justification for the instrument outlined in JWX. 

For certain analyses, JWX uses 2008 state K-12 revenue share to group states 
into high-, medium-, and low-reliance categories. Because of the data differences 
discussed above, applying the same categorization rules to different data sources 
yields different groupings. Table 2 includes color coding to indicate high- and low- 
reliance groups across data sources. Cells colored green are states which are below 
the 0.33 threshold JWX uses to define low-reliance, and cells colored red are above 
the 0.67 threshold JWX uses to define high-reliance. We will discuss these 
thresholds in more detail in the next section. Simply changing data sources from 
JWX to NCES results in four states changing categories and D.C. dropping from 
the analysis. While this change only represents approximately 10 percent of 
observations, as we will show in the next section, these five states represent a very 


high proportion of JWX’s high- and low-reliance groups. 
State-Group Definitions 


The primary analysis in JWX relies on grouping states into high-, medium-, and 


low-reliance categories. In addition to the data issue outlined in the previous 


'9 JWX provides estimates dropping Washington D.C., Hawaii, and/or California, and finds that their results are robust to 
these changes. 
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section, the thresholds JWX use to categorize states result in very few states being 
placed in the high- and low-reliance categories. JWX does not provide a theoretical 
or empirical justification for the thresholds used to categorize states and does not 
test different groupings. 

As noted in the previous section, JWX defines high-reliance states as those that 
depend on state appropriations for more than 67 percent of total revenue and low- 
reliance as less than 33 percent. Of the 51 jurisdictions included in the analysis (1.e., 
50 states and Washington D.C.), the JWX categorization rules result in only three 
states being placed in the low-reliance group and four in the high reliance group. 
All other states are relegated to the middle-reliance category. 

Of the seven states included in the high or low groupings, four of them are based 
on questionable data, including three of the four states in the high-reliance category. 
Outlier jurisdictions Washington D.C. (low-reliance with zero percent state share) 
and Hawaii (high-reliance with 85 percent state share) are included in the low- and 
high-reliance categories, respectively.”° In addition, Arkansas (high-reliance with 
76 percent state share) and Vermont (high-reliance with 68 percent state share), 
who’s property tax revenues are arguably misclassified as state revenue, are 
included in the high-reliance category. 

The small number of states in the high- and low-reliance groups and the presence 
of states with problematic data in both groups raises the question of how sensitive 
the JWX results are to state-group definitions. JWX provides results from 
robustness tests dropping any combination of one, two, or three states from their 
analysis (see figure A3 in the online appendix). However, this does little to test the 
robustness of their results to different state groupings given that the vast majority 


of states (1.e., 44) are in the middle-reliance category. The results from the test JWX 


?° The school districts in both Washington D.C. and Hawaii combine state and local government levels, making attempts 
to separate state vs. local revenue challenging. 
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performs would primarily be generated by dropping states in the large middle 
reliance category, which we would expect to have a limited effect on the estimate. 

We test the impact of state-group definitions by estimating results using four 
symmetric percentile sets to categorize states, as depicted in the columns of Table 
3. We use average 2005-07 state K-12 revenue share calculated using NCES data 
to determine the percentiles and categorize states. Column 1 of Table 3 depicts the 
grouping used in JWX, and columns 2-5 represent additional groupings based on 
the 10" and 90" percentile, 15 and 85" percentile, 20 and 80" percentile, and 
25" and 75" percentile, respectively. 

In Table 4, states highlighted in red text are those which are different from the 
JWX grouping (Column 1). States not listed in either the low- or high-reliance 
groups are in the medium-reliance category. The number of states in each category 
is listed in parenthesis at the top of each column. Increasing the threshold for low- 
reliance from the JWX value (0.33 percent) to the 10", 15%, 20" and 25% 
percentiles adds 2, 4, 8, and 10 states to that group, respectively. 

Figure 3 provides a histogram illustrating the distribution of our pre-recession 
state revenue share measure. The red lines represent the various percentiles we use 
to categorize states. Most states are relatively closely clustered between 30 percent 
and 65 percent, with only six states outside that range. For illustrative purposes 
Washington D.C. is included in the figure with a value of zero (i.e., the value from 
JWX). Washington D.C. is the bar on the far left at zero, more than 25 percent 
below the next state. Hawaii and Vermont are the two outliers on the right side of 
the figure. 

Given the variation in group composition with different threshold rules and data 


sources, our empirical approach investigates how state group definitions influence 


?! The percentile values for NCES average 2005-07 state K-12 revenue share rounded to four digits are: 10" = 0.3364, 
15" = 0.3585, 20" = 0.3862, 25" = 0.4000, 75" = 0.5746, 80" = 0.5937, 85" = 0.6031, and 90" = 0.6273 
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the JWX results. We believe using percentiles to define state groups is empirically 
justifiable and testing more inclusive groupings will help us better understand the 
influence of the groupings on the results. Sections IV and V outline our analytical 
approach and present findings for different specifications using each grouping 


included in Table 4. 


Controlling for Year-Specific Shocks 


The Great Recession was a complex event with multiyear impacts on many 
aspects of our society. Given the nature of the identifying event, annual stochastic 
shocks may influence both the outcome measures and the instrument. In 
circumstances such as these it is common practice to implement individual year 
fixed effects in empirical models to control for potential annual shocks. 

JWX employs the full set of year fixed effects in models that group states into 
reliance categories (i.e., the group IV models). However, as the previous section 
outlines, we have reason to believe the results of those models may be sensitive to 
state-group definitions. The appendix of JWX also presents models that do not rely 
on state groupings, but instead include 2008 state revenue share as a continuous 
variable (i.e., linear IV models; see section C.2 of the JWX online appendix). JWX 
provides these linear models to demonstrate the robustness of the results. 

The authors note that including the full set of year fixed effects in the linear 
specification would be ideal but doing so results in a “weak first stage” (footnote 
18 on p. 11 of JWX, 2019 and IV.1 section F of JWX, 2020a). We test this claim 
and find that when the full set of year fixed effects is included, the first stage 
coefficient estimate is close to zero and the Kleibergen-Paap Wald F-statistic is 
lower than the critical value, thus validating that the instrument is weak in the 


presence of full year fixed effects. 
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In lieu of using full year fixed effects, JWX’s preferred linear specification (see 
Table A23 of the JWX online appendix) groups years into three period controls 
representing pre-recession (2001/02-2007), during-recession (2009-2011), and 
post-recession (2013-2017). There are at least two motivations to test for sensitivity 
to different specifications of year fixed effects. First, there could be factors related 
to the recession that vary with year and affect the outcomes through channels other 
than school spending. Grouping years will not fully account for these factors. 

Second, imprecision around the timing of the recession and its impacts 
recommends caution when grouping years together over this timeframe. The 
National Bureau of Economic Research (NBER) estimates that the Great Recession 
began in December of 2007 and ended in June of 2009.” Of course, in most states, 
the budget implications of the downturn lasted beyond the official end of the 
recession. JWX present models using only the year groupings described above. 
Given the uncertainty around how best to control for year-specific shocks, we test 
six different reasonable configurations of the year fixed effects in our analysis. Our 


approach is outlined in section IV. 
Potential Confounding of the JWX College-Going Result 


For K-12 state revenue share to be a valid instrument for PPE, it must not affect 
the college-going rate through any other pathway except through PPE, and it must 
not be influenced by any other factors which are also causally linked to college- 
going. We believe that potential confounding between K-12 and higher education 
funding trends may violate the exclusion restriction necessary to causally estimate 
the effect of PPE on college-going rates. We also believe there may be economic, 


political, and institutional factors that influence both pre-recession K-12 state 


» See https://www.nber.org/cycles.html. 
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revenue share and post-recession college-going rate. Figure 4 provides a directed 
acyclic graph (DAG) illustrating our concerns. 

To investigate confounding, we first examine time trends to see if K-12 and 
higher education funding measures move in tandem. Figure 5 depicts the 
relationship among PPE, state and local higher education appropriations per full- 
time equivalent (FTE) student, and tuition revenue per FTE. Per-pupil expenditures 
and higher education appropriations move closely together after the recession, both 
falling sharply between 2008 and 2012, while tuition revenue steadily increases 
over time. Figure 6 shows these same three variables indexed to their 2002 values 
to display percentage change. The magnitude of the changes in higher education 
appropriations and tuition dwarf those for PPE, raising the prospect that higher 
education finance might have greater potential to influence college-going rates. 

These graphs lead us to believe that K-12 and higher education spending may be 
confounded and even if a relationship between K-12 PPE and college-going rates 
exists, it may be overshadowed by the impact of changing higher education 
appropriations and tuition revenue. It is reasonable to theorize that higher education 
funding may influence college-going more than K-12 appropriations. Recent 
studies have shown that college outcomes, including enrollment, are responsive to 
higher education funding changes (Deming and Walters, 2017 and Bond et al., 
2019). While we do not attempt to identify the true effect of higher education 
spending on college enrollment, we investigate the possibility that higher education 
appropriations and tuition confound the JWX college-going findings. Section IV 


provides additional details regarding our approach. 


IV. Empirical Approach 


Our empirical methods follow JWX closely, with variable definitions matching 


those described in the Identification Strategy section of JWX. To our knowledge, 
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we deviate only in areas where we test the robustness of the JWX results. The 
sections below present our empirical approach for four different analyses: 1) event 
study graphs, 2) group IV, 3) linear IV, and 4) tests for higher education 
confounding. Each section describes our methods and approach for robustness 


checks.” 
Event Study Graphs 


JWX presents event study graphs as suggestive evidence that per pupil 
expenditures, NAEP scores, and the college-going rates declined more in states 
with a high reliance on state revenues relative to states with a low reliance on state 


revenue. Equation 1 describes the event study estimating equation. 


(1) Vor = Dt23003 Be (lo,>q(s0),s X Irae) + @s + 04 + (Ts X T) + Us¢ 
Subscripts s and ¢ indicate state and year, respectively. The outcome variable, Y5,, 
is either PPE, average NAEP scores, college-going rates, or high school graduation 
rates. Ig +4(50),s 18 an indicator for states where the 2008 K-12 state share is above 
the national median of 0.48. We use the JWX state share data for this analysis. p= 
are year-specific indicators. The a, are state fixed effects, and the 6; are year fixed 
effects. T, is the pre-recession, state specific linear trend for the left-hand side 
variable in each equation.” , is the coefficient of interest and represents the year- 
specific difference between states above versus below the median. We omit the year 


2007, so the calculated differences are relative to that year. 


°3 Errors are clustered at the state level in all models. 

4 These linear trends are equal to the slope of the left-hand-side variable between 2001/2002 and 2007. We also ran 
models where we instead used regression to predict the left-hand-side variables based on pre-recession data with similar 
results. 
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Instrumental Variables Models 


Following JWX, we estimate two different sets of IV models. The first is a group 
IV in which states are categorized as being high-, medium-, or low-reliance based 
on state share of K-12 revenue. These category variables are then used to define the 
instrument. The second model is a linear IV in which state share of education 
revenue is treated as a continuous variable. While the forthcoming version (Jackson 
et al., 2020a) focuses exclusively on the group specification, the Northwestern 
University working paper version of JWX (Jackson et al., 2020b) includes both the 
linear and group IV specifications. We include both models to test the robustness 


of the results across multiple dimensions. 


Group IV.—For the group IV analysis, we estimate the equations below using two 


stage least squares (2SLS). 
(2) PPEse = Y3-2 [1g (Igs X Ipose X (T — 2008))| + 03-2[O19(lgs X Ipost)] + 
P12 Case) + OC +05 +6¢+ (15 XT) + 


(3) Yor = B(PPE.t) + Yo =2[919 (gs x Tyost) | + P22 (ast) + 6€C 5, + 25 + A, + 


(t2;XT) +e 
The subscript g refers to the reliance categories, with 1 representing low-reliance, 
2 representing medium-reliance, and 3 representing high-reliance. I, is a state 
specific indicator for group membership, J,,,;¢ is an indicator for years after 2008, 
and (T — 2008) indicates current year, T, relative to 2008. C,, is a vector of state 
and year-specific Bartik economic controls for the unemployment rate and average 
annual wage, which are defined in JWX. State (a@,) and year (6,,) fixed effects are 


included, along with the pre-recession, state-specific linear trend for the left-hand- 
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side variables (t,,). The second stage outcome variable, Y,,, is either a) average 
standardized NAEP scores, b) college-going rate, or c) high school graduation 
rates.*> B, the coefficient on the predicted values from the first stage, PPE,,, is the 
coefficient of interest. 

To test the impact of state group definitions on the results, we vary state-reliance 
group membership as described in the motivation section above and outlined in 


Table 4. 


Linear IV.—For the linear IV, we estimate the model described below using 2SLS. 
(4) PPE ce = 1 (M5 X Ipose X (T — 2008)) + ra (Ms X Ipose) + P12(Ipose) + 

6C 5 +0,,+6;,+(%,xXT)+€ 
(5) Yt = B(PPEst) 7 P21 (Os x Tose) Sa P22( post ) + OC + A5 + O24 + 


(T2,XT) +e 

All variable definitions are the same as in the group IV described above. 
However, in this model the reliance group indicators are replaced with the 
continuous variable 0, which is 2008 K-12 state revenue share. We use the JWX 
state share data for this analysis. 

Since there is uncertainty about how best to control for year-specific shocks, we 
test the linear model using seven different specifications for year fixed effects. 
Table 5 describes each of the specifications we test. Column 2 provides the number 
of year indicator variables included in the model in each specification. We 
concentrate our robustness tests on years around (2007-2009) and those after (201 1- 


2017) the Great Recession. 


°5 High school graduation rate models include an indicator for the change from AFGR to ACGR. 
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Higher Education Confounding 


We initially test for confounding by adding pre-recession state higher education 
revenue share, state and local higher education appropriations per FTE, and tuition 
and fees per FTE into the linear IV as control variables. However, adding these 
variables as controls creates a multicollinearity problem. The first-stage variation 
inflation factors (VIFs) for all three variables are well above recommended levels, 
indicating significant multicollinearity with other included variables. In fact, the 
statistical software drops pre-recession state higher education revenue share from 
linear IV models because of collinearity when it is included as a control. Therefore, 
we proceed by testing confounding in the three ways described below using the 
linear IV model outlined above. 

First, we replace the K-12 instrument with pre-recession state share of higher 
education revenue. If pre-recession state share of higher education revenue has 
similar predictive power to K-12 state revenue share, then state funding for K-12 
and higher education may be confounded. 

Next, we estimate the linear IV described above, but replace the dependent 
variable with state and local higher education appropriations per FTE and tuition 
and fees per FTE. If instrumented PPE predicts these higher education funding 
variables, then the exclusion restriction may be violated in the JWX model. 

JWX provides results from a similar robustness test (see Table A18 in the online 
appendix). However, there are two key differences between our test and theirs: 1) 
we use per FTE measures analogous to per-pupil spending for K-12 and 2) we 
perform these robustness checks using the linear IV in which the instrument is a 
continuous variable rather than based on discrete state groups. Using a per FTE 
adjusts for state size, similar to K-12 spending, while using the linear IV model 


eliminates the influence of state groupings on the robustness test. 
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Finally, we estimate a JWX-style linear IV predicting college-going using state 
share of higher education to instrument for state and local higher education 
appropriations per FTE and tuition and fees per FTE. If a similarly specified IV 
model using higher education variables can predict college-going, then it indicates 


that higher education is a potential alternative pathway influencing college-going. 
V. Results 
Event Study Graphs 


Figures 7-10 provide the event study graphs from our analysis. These figures 
include a dot indicating the point estimate for the year-specific difference between 
states that are above versus below the median for 2008 K-12 state revenue share, 
as well as 95 percent confidence intervals around the point estimates. Point 
estimates are relative to 2007, the omitted year in the model. 

We find evidence that, following the recession, PPE declined more in states 
where pre-recession K-12 state revenue share was above the median (Figure 7). 
However, for NAEP scores (Figure 8), college-going rates (Figure 9), and high 
school graduation rates (Figure 10) we find no statistically significant differences 
between states that are above versus below the median K-12 state revenue share. 

Our event study results are different from JWX, which finds significant 
divergence after 2007 in NAEP scores and college-going rates for states that are 
above versus below the state revenue share median. The threshold used in JWX for 
the event study graphs has changed significantly across different versions of the 
paper (i.e., from 0.33 to 0.39 to 0.48 in the current version). Based on our analysis, 
we believe that the latest version of JWX may present results for a lower cutoff 


(i.e., 0.39) than is stated in the paper (i.e., 0.48 - the national median).”° 


°6 The change in the cutoff from 0.39 to 0.48 moves 16 states from one group to the other. 
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Group IV 


Tables 6A through 6C present the results of our group IV estimation. The letters 
A-C in the table names correspond to our three outcome variables: A) NAEP scores, 
B) college-going rates, and C) high school graduation rates. We estimate results 
both including (Columns 2-6) and excluding (Columns 7-10) Washington D.C. 
Column | provides the original results from JWX for comparison. We model each 
of the five state categorization frameworks presented in Table 4: the JWX 
categorization (Column 2 in Tables 6A-C) as well as categorizations using NCES 
data for state K-12 revenue share based on the 10" and 90" percentiles (Columns 3 
and 7), 15th and 85" percentiles (Columns 4 and 8), 20" and 80" percentiles 
(Columns 5 and 9), and 25th and 75" percentiles (Columns 6 and 10).”’ 

For both NAEP (Table 6A) and college-going (Table 6B), our replication 
estimates using the JWX categorization (Column 2) are similar in magnitude and 
significance to the original JWX results (Column 1). When we apply other 
categorizations based on percentiles of state K-12 revenue share, the point estimates 
shrink toward zero and lose significance. In the NAEP models, all estimates using 
percentiles to categorize states are statistically insignificant, and in our preferred 
specifications, which excludes Washington D.C., the point estimates switch sign 
when using the bottom and top 15 percent or greater to categorize low/high-reliance 
states. For college-going, the point estimates are somewhat more stable — although 
attenuated and insignificant in all percentile-based specifications. We also provide 
results for high school graduation rates (Table 6C), which are insignificant across 
all specifications. 

Our estimates indicate that the JWX group IV results are sensitive to changes in 


state group composition. Using the top and bottom decile of pre-recession state K- 


7 For NCES state share, we use average state share from 2005-2007 instead of 2008 state revenue share. This ensures 
that the instrument is measured pre-recession and captures persistent state share. 
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12 revenue share only changes two states in the low-reliance group and three states 
in the high-reliance group relative to the JWX categorization, but even this small 
change yields statistically insignificant results for both NAEP and college-going. 
Given that the point estimates when using top/bottom decile are not statistically 
different from the original JWX results, we cannot rule out the possibility that there 
is an effect at extreme values of the instrument for both outcomes. However, the 
NAEP results strongly suggest that any potential effect on achievement dissipates 
relatively quickly as group composition becomes less extreme. The college-going 
point estimates are somewhat more persistent, but we are concerned that these 
results may be confounded by higher education funding. In a subsequent section, 
we present evidence for the potential confounding of the college-going results with 


contemporaneous higher education funding trends. 


Linear IV 


As noted in the motivation section, a key component of the linear IV specification 
is how year fixed effects are modeled. JWX groups years into three indicator 
variables for pre-, during-, and post-recession. To better understand the model’s 
robustness around this choice, we test different specifications for year fixed effects. 
Table 5 describes each of the specifications we employ. 

Tables 7A-C present the results of our linear IV. As in the previous section, the 
letters in the table names correspond to our outcomes: A) NAEP scores, B) college- 
going rates, C) high school graduation rates. Column | provides the original JWX 
results for comparison, and Column 2 provides our replication results for the JWX 
specification. The remaining columns (3-9) in Tables 7A-C correspond to rows in 
Table 5, which outline the various year fixed effect specifications we test. We 


exclude Washington D.C. from the models presented in columns 3-9. 
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We replicate similar magnitudes and significance levels for the impact of 
instrumented PPE on both NAEP and the college-going rate when using the JWX 
year-group specification (Columns 1 and 3). For NAEP scores, all other 
specifications for the year fixed effects yield insignificant results and point 
estimates that are negative or zero (Table 7A). On the other hand, the findings for 
college-going rate appear to be robust to changes in year fixed effects specification 
as all specifications, except full year fixed effects, yield positive and statistically 
significant results of roughly the same magnitude (Table 7B). The point estimates 
for high school graduation are very close to zero and not consistent in sign or 
statistical significance. 

We also run models using pre-recession property tax revenue share as the 
instrument. The estimates generated by these models are provided in Tables 8A-C. 
Using property tax revenue share to instrument for PPE produces similar estimates 
to those generated using state revenue share. For NAEP, the results using the JWX 
year grouping produces positive significant results; however, the other year 
groupings produce negative point estimates with inconsistent statistical 
significance. College-going estimates again appear relatively robust to different 
specifications for year-specific controls. Estimates for high school graduation rates 
are close to zero and not consistent in sign or statistical significance. 

These findings indicate that the NAEP results are highly sensitive to different 
specifications of the year fixed effects. Estimates generated using the specific year 
grouping employed in JWX are relatively robust to dropping Washington D.C. 
from the analysis and using property tax share for the instrument. However, all 
other reasonable year fixed effect specifications we test yield point estimates that 
are either negative or zero. The weakness of the first-stage is a likely driver of this 


result - F statistics from the first stage of alternate year fixed effect specifications 
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indicate a weak instrument. 78 Given there is not a strong theoretical reason to group 
years in the specific way JWX implements, and since we find many other 
reasonable specifications yield a weak first stage and null results, we conclude that 
the JWX linear IV model is unlikely to identify a systematic relationship between 
NAEP scores and spending cuts following the Great Recession. 

While the college-going results appear more robust, the sensitivity of the NAEP 
results and lack of statistically significant effects on high school graduation rates 
call into question the pathway by which PPE affects college-going. If PPE did not 
have a consistent effect on student achievement and did not impact graduation rates, 
then how did it affect the college-going rate? The next section provides one 


potential answer to that question. 
Higher Education Confounding 


While the JWX college-going results appears to be somewhat robust in models 
presented above, there is reason to believe that the finding may be confounded with 
contemporaneous higher education trends. Section III describes why we believe 
this is the case and section IV lays out our three-pronged approach for empirically 
demonstrating that confounding likely exists between K-12 and higher education 
spending. 

First, we replace the K-12 state share instrument with pre-recession state share of 
higher education revenue. Table 9 presents the results of this exercise for both 
NAEP and college-going rates. The first two columns in each panel provide the 
original JWX findings and our replicated findings from Table 7. Results using the 
higher education state share instrument are found in the final two columns of Table 


9. Our findings indicate that replacing the K-12 state share instrument with higher 


8 Recent analysis (Lee et al. 2020) suggests that second-stage t-statistic critical values may be more sensitive to the 
strength of the first stage than previously thought, and as a result, the statistical significance of many IV results may be 
overstated. 
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education revenue yields point estimates that are of similar magnitude and 
significance to those obtained using K-12 state share. This indicates that state shares 
of K-12 and higher education revenue are likely confounded. 

Next, we estimate linear IV models replacing the dependent variable with state 
and local higher education appropriations per FTE and tuition and fees per FTE. 
Tables 10A and 10B present the results of those models. As in the linear IV section 
we provide results across all year fixed effect specifications outlined in Table 5. 
For all 16 specifications, except the model including full year fixed effects for 
tuition and fees, we find that instrumented PPE is statistically significant and 
positively related to state and local higher education appropriations per FTE and 
tuition and fees per FTE. The ability of instrumented PPE to predict higher 
education funding variables indicates that the IV exclusion restriction may be 
violated in the JWX college-going model. 

Finally, we estimate JWX-style IV models to predict college-going rates using 
pre-recession state share of higher education revenue to instrument for state and 
local higher education appropriations per FTE and tuition and fees per FTE. We 
also estimate these models substituting K-12 state share as the instrument in place 
of higher education revenue. This investigates whether the confounding of the 
instrument is also apparent when higher education spending is treated as the 
endogenous variable. We estimate these models using the natural log of 
appropriations and tuition and fees per FTE. Table 11 provides the results for these 
models. All specifications yield statistically significant, positive point estimates, 
indicating that higher education spending is a viable alternative pathway for 
impacting college-going rates. 

Based on the results of the three tests, we believe that the college-going finding 
is confounded with contemporaneous higher education trends. While it may be 


feasible that both K-12 spending and higher education spending impact college- 
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going, we do not think it is possible to disentangle those impacts using a JWX-style 


IV approach. 


VI. Conclusions 


As states face looming budget shortfalls and schools grapple with the effects of 
COVID-19, it has become increasingly important to understand the potential 
impact of education funding reductions on student outcomes. Using data from the 
years around the Great Recession, JWX suggests that K-12 spending reductions 
have widespread negative impacts on both student achievement and attainment. 

While we believe that spending reductions may have negative impacts on student 
outcomes, our results suggest that there is not a clear and compelling story about 
the impact of school spending reductions related to the Great Recession on student 
achievement. Moreover, we find that the relationship between K-12 spending and 
college-going rates is likely confounded with contemporaneous higher education 
funding trends. Specifically, we find: 

e the JWX group IV model categorizes very few states as having low- 
reliance (three states) or high-reliance (four states) on state revenue to 
fund K-12 education, and while there may be an effect at the extremes 
of the distribution, the results are sensitive to relatively small changes 
in group composition; 

e the JWX linear IV uses a non-standard specification for year fixed 
effects, and the results are sensitive to using other reasonable 
specifications to control for year-specific shocks; and 

e post-recession K-12 and higher education funding trends are related, 
and as a result, the JWX finding that K-12 spending reductions 
impacted state college-going rates is likely confounded with 


contemporaneous higher education funding trends. 
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Our results highlight the challenges of estimating causal effects for loosely 
defined interventions (e.g., funding changes) at the level of states and the 
importance in exercising caution when making policy recommendations based on 
such studies. It is not surprising that it is difficult to empirically estimate the impact 
of school funding on student outcomes. K-12 education is a complex system and 
how schools deal with funding changes varies widely both within and across states. 
Unless those changes are substantial and have a direct, sizeable impact on 
classrooms, student outcomes are unlikely to respond quickly enough to be 
captured in short timeframes. Therefore, it is questionable whether rigorous 
research can consistently show that school spending matters for educational 


outcomes on such a broad scale. 
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Tables and Figures 


Figure 1: Per-Pupil Current Expenditures (PPE) and Average NAEP Scores 
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Notes: PPE was collected from the 2019 NCES Digest of Education Statistics table 236.55. Average NAEP is the national 
public average score on reading and mathematics tests standardized to 2003. NAEP data were collected from the NAEP Data 
Explorer. Only the reading test was administered in 2002. Mathematics test scores for that year were imputed using linear 
interpolation 


Figure 2: Per-Pupil Current Expenditures (PPE) and College-Going Rate 
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Notes: PPE was collected from the 2019 NCES Digest of Education Statistics table 236.55. The college-going rate is the 
number of first-time enrollees in the fall semester who graduated high school in the past 12 months divided by the Census 
estimate for the number of 17-year-olds in the previous year. The national rate was calculated by summing the numerator 
and denominator across states in each year. See the Data section for more details on the data and calculation. 
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Figure 3: Histogram of 2005-07 Average State K-12 Revenue Share 
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Notes: The red lines represent the percentile labeled at the top of the figure. For illustrative purposes, Washington D.C. is 
included in the figure with a value of zero despite being listed as N/A in the NCES data. 


Figure 4: College-Going IV Directed Acyclic Graphs (DAGs) 
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Figure 5: Per-Pupil Current Expenditures (PPE) and 
Higher Education Funding Trends 
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Notes: PPE was collected from the 2019 NCES Digest of Education Statistics table 236.55. State and local appropriations 
per FTE and tuition and fees per FTE are obtained from the State Higher Education Executive Officers Association (SHEEO 
2019). 


Figure 6: 2002 Indexed Per-Pupil Current Expenditures (PPE) and 
Higher Education Funding Trends 
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Notes: PPE was collected from the 2019 NCES Digest of Education Statistics table 236.55. State and local appropriations 
per FTE and tuition and fees per FTE are obtained from the State Higher Education Executive Officers Association (SHEEO, 
2019). 
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Figure 7: Per-Pupil Expenditures Event Study Graph 
Using 48 Percent Threshold for High-Reliance 


an 


0 


-1 
1 


Thousands of Dollars 


2001 2002 2003 2004 2005 2006 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 
Year (base of 2007) 


Figure 8: NAEP Scores Event Study Graph 
Using 48 Percent Threshold for High-Reliance 
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Figure 9: College-Going Rate Event Study Graph 
Using 48 Percent Threshold for High-Reliance 
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Figure 10: High School Graduation Rate Event Study Graph 
Using 48 Percent Threshold for High-Reliance 
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Table 1: Study Data Summary Statistics 


JWX Values Replication Values P-Value for t-test 
N Mean SD N Mean SD of Difference in 
Means 
Per-pupil spending 510 13,208 3,807 510 13,203 3,699 0.98 
(2015 dollars) 
Share of revenue from state 51 0.493 0.137 51 0.493 0.137 
sources in 2008 (JWX) 
Share of revenue from state 51 0.502 0.144 
sources in 2008 (Census) 
Share of revenue from state 51 0.496 0.144 
sources in 2008 (NCES) 
Share of revenue from 51 0.300 0.131 
property taxes in 2008 (NCES) 
Average NAEP Z-Score 459 0.125 0.211 459 0.120 0.200 0.71 
(standardized to 2003) 
College Enrollment Rate 459 0.475 0.078 459 0.480 0.079 0.33 
(17-year-olds) 
High School Graduation Rate 510 0.773 0.081 
Unemployment Rate 510 5.777 1.965 510 5.742 1.955 0.78 
Annual Average Employment 510 | 2,603.81 2,820.57 510 2,600.10 2,822.63 0.98 
(thousands) 
Average Annual Wage 510 53,052.66 10,744.91 
(2015 dollars) 
Bartik Unemployment Rate 510 5.972 1.659 
Bartik Average Annual Wage 510 46,807.09 1,496.35 
(2015 dollars) 


Notes: Variables were collected for each state and the District of Columbia for the following years: 2001, 2002, 2003, 2005, 
2007, 2009, 2011, 2013, 2015, and 2017. College-going variables are available for odd numbered years starting in 2001. We 


use NAEP from 2002 matching JWX. 


NABFP test scores are standardized to 2003, and then averaged for each state and year. Only the reading test was administered 
in 2002, and five states do not have 2002 NAEP scores: AK, CO, NH, NJ, and SD. We used 2003 average reading z-score 


as the 2002 value for these five states. 


We model the college-going rate as the number of first-time college enrollees divided by the number of 17-year-olds in a 


state. 


Data from 2001-2013 represent Average Freshman Graduation Rate (AFGR) values and data from 2014-2019 are Adjusted 
Cohort Graduation Rates (ACGR). 
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Table 2: Data on State and Property Tax K-12 Revenue Share 


(1) (2) (3) (4) 
State 2007-10 (3yr) 2007-12 (Syr) 2007-17 (10yr) 
JWx Real Per-Pupil Real Per-Pupil Real Per-Pupil 

Categorization | Expenditure Change | Expenditure Change | Expenditure Change 
Alabama M 317 -912 -242 
Alaska M 3,515 4,536 3,646 
Arizona M 260 -819 -717 
Arkansas 607 216 93 
California M 109 -510 2,084 
Colorado M 88 -824 122 
Connecticut M 1,305 1,899 4,090 
Delaware M -540 215 868 
District of Columbia L 2,185 17 1,979 
Florida M -293 -1,413 -1,245 
Georgia M -412 -1,365 -737 
Hawaii -281 -1,166 655 
Idaho M -34 -939 -402 
Illinois iL, 1,870 1,560 4,671 
Indiana M -195 -693 -1,138 
Iowa M 1,394 1,058 1,795 
Kansas M -64 -831 -1,107 
Kentucky M 583 367 141 
Louisiana M 1,456 661 605 
Maine M 798 -85 1,140 
Maryland M 1,530 487 757 
Massachusetts M 644 915 3,223 
Michigan M 64 -807 -570 
Minnesota M 646 719 1,451 
Mississippi M 221 -313 -29 
Missouri M 736 45 535 
Montana M 686 -144 326 
Nebraska L 641 193 1,388 
Nevada M -23 -730 -346 
New Hampshire M 1,609 1,743 2,998 
New Jersey M 447 -404 985 
New Mexico 342 -919 -597 
New York M 2,014 1,923 4,234 
North Carolina M 47 -563 -256 
North Dakota M 1,408 1,540 3,657 
Ohio M 992 268 199 
Oklahoma M 60 -669 -1,047 
Oregon M 130 -552 TAT 
Pennsylvania M 1,102 669 2,682 
Rhode Island M 715 155 507 
South Carolina M 106 -515 353 
South Dakota M 762 -430 829 
Tennessee M 564 547 817 
Texas M 609 -662 188 
Utah M 448 -34 649 
Vermont 1,741 1,919 4,203 
Virginia M -238 -868 -456 
Washington M 512 133 2217 
West Virginia M 1,958 828 695 
Wisconsin M 834 -338 -144 
Wyoming M 1,681 1,534 1,094 


Note: In column 1, green is used to highlight states are categorized as low-reliance in JWX and red is used to highlight states 


that are categorized as high-reliance in JWX. Constant dollar change in per-pupil expenditures was collected from the NCES 


Digest of Education Statistics Table 236.70. 
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Table 3: Data on State and Property Tax K-12 Revenue Share 


dd) (2) (3) (4) (5) 

JIWX Census NCES NCES NCES 

State Abbreviation 2008 State Revenue Share Avg. 2005-07 State | Avg. 2005-07 Property 
Revenue Share Tax Revenue Share 

Alabama AL 60.2% 60.2% 60.6% 56.2% 13.0% 
Alaska AK 64.9% 64.9% 66.3% 58.7% 9.8% 
Arizona AZ 48.1% 48.5% 51.7% 49.2% 29.8% 
Arkansas AR 56.7% 57.5% 25.1% 
California CA 57.9% 59.9% 61.3% 60.0% 21.6% 
Colorado CO 42.1% 42.4% 42.2% 42.8% 38.7% 
Connecticut CT 37.8% 38.5% 39.6% 38.5% 54.7% 
Delaware DE 61.2% 63.0% 62.0% 62.7% 23.3% 
District of Columbia DC 0.0% 0.0% N/A N/A 21.3% 
Florida FL 39.4% 39.4% 38.8% 40.8% 39.9% 
Georgia GA 45.1% 45.2% 45.4% 44.4% 31.3% 
Hawaii HI 0.0% 
Idaho ID 65.5% 65.5% 60.3% 24.3% 
Illinois IL 32.9% 33.8% 31.2% 30.6% 51.7% 
Indiana IN 47.3% 48.5% 53.5% 49.8% 31.3% 
Iowa IA 44.8% 46.5% 46.5% 45.6% 31.7% 
Kansas KS 58.4% 58.4% 57.5% 55.6% 26.1% 
Kentucky KY 57.9% 57.9% 57.3% 56.9% 20.9% 
Louisiana LA 43.6% 43.9% 44.8% 44.5% 14.2% 
Maine ME 43.2% 44.5% 44.9% 42.9% 44.8% 
Maryland MD 42.0% 42.0% 42.1% 39.1% 25.3% 
Massachusetts MA 41.8% 42.1% 41.9% 45.4% 45.1% 
Michigan MI 54.6% 57.3% 57.5% 59.4% 27.0% 
Minnesota MN 64.4% 65.8% 65.9% _ |NOODUOE 13.7% 
Mississippi MS 53.7% 53.8% 54.5% 52.8% 22.8% 
Missouri MO 40.8% 41.1% 33.3% 33.6% 43.0% 
Montana MT 49.0% 49.4% 49.7% 46.5% 24.2% 
Nebraska NE 32.3% 33.0% 33.1% 31.6% 50.4% 
Nevada NV 57.5% 57.5% 30.8% 26.6% 28.4% 
New Hampshire NH 37.1% 38.6% 38.6% 38.6% 52.1% 
New Jersey NJ 40.0% 41.3% 42.1% 42.6% 49.2% 
New Mexico NM [| 71.2% | 71.2% | 708% | 709% | 10.2% 
New York NY 45.2% 45.4% 44.8% 43.0% 45.0% 
North Carolina NC 58.8% 58.8% 65.7% 62.9% 20.6% 
North Dakota ND 34.7% 36.1% 36.3% 36.2% 37.9% 
Ohio OH 43.0% 44.1% 45.6% 44.0% 39.5% 
Oklahoma OK 51.2% 51.2% 54.2% 53.5% 22.7% 
Oregon OR 52.4% 52.8% 52.3% 50.2% 29.2% 
Pennsylvania PA 34.3% 35.8% 36.5% 35.9% 41.7% 
Rhode Island RI 38.5% 38.7% 39.9% 40.0% 50.3% 
South Carolina SC 50.6% 50.7% 50.8% 44.9% 34.3% 
South Dakota SD 33.1% 33.2% 33.9% 33.1% 42.0% 
Tennessee ™ 45.9% 46.1% 45.6% 43.0% 21.4% 
Texas TX 43.1% 43.2% 44.8% 35.8% 46.4% 
Utah UT 56.3% 56.3% 56.7% 55.3% 26.1% 
Vermont VT | 683% | 885% | 859% | 855% | 00.2% 
Virginia VA 40.3% 41.0% 41.0% 40.6% 23.9% 
Washington WA 61.9% 62.4% 62.5% 60.9% 23.2% 
West Virginia WV 58.1% 58.1% 59.1% 60.0% 24.4% 
Wisconsin WI 49.2% 50.1% 50.0% 51.6% 37.8% 
Wyoming WY 52.8% 52.9% 52.8% 48.3% 28.7% 


Note: Census values are provided because JWX differs from the Census data. Green is used to highlight states that have 2008 
state revenue share less than 0.33 and red is used to highlight states that have 2008 state revenue share greater than 0.67. 
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Table 4: Categorizations for Group IV 


() (2) (3) (4) (5) 
JWx 10" and 90" 15" and 85" 20" and 80" 25" and 75" 
Groups Percentiles Percentiles Percentiles Percentiles 
Low-Reliance (3 States) (5 States) (7 States) (11 States) (13 States) 

D.C. D.C. D.C. CT CT 
IL IL IL D.C. D.C. 
NE NE MO IL IL 
NV NE MO MD 
SD NV NE MO 

SD NV NE 

TX NH NV 

ND NH 

PA ND 

SD PA 

TX RI 

SD 

TX 

High-Reliance (4 States) (5 States) (7 States) (10 States) (12 States) 

AR HI DE CA AK 
HI MN HAI DE CA 
NM NM MN HI DE 
VT NC NM ID HI 
VT NC MN ID 

VT NM MI 

WA NC MN 

VT NM 

WA NC 

wv VT 

WA 

wv 


Notes: Percentile groups are determined using NCES average 2005-07 state revenue share. States that are highlighted in 


red 


are different from the JWX categorization (column 1). D.C. and Hawaii are in bold and italics because their education funding 
systems are fundamentally different from other states. We drop D.C. from some analyses. Dropping D.C. results in MO 
moving into the bottom decile and PA into the bottom 15" percentile, otherwise the groups remain the same. Based on the 
text of JWX the group cutoffs are state share less than 1/3 for low reliance and state share greater than 2/3 for high reliance. 
When implementing these rules, JWX appears to have simplified to 0.33 and 0.66. If the 1/3 rule had been applied strictly, 
South Dakota should also have been included in the low-reliance group. 


Table 5: Year Grouping 


s for Fixed Effects 


Number of | 2001/2002 | 2003 | 2005 | 2007 | 2009 | 2011 | 2013 | 2015 | 2017 

Year Fixed 

Effects 
JWX Grouping 3 
Grouping 2 3 
Grouping 3 4 
Grouping 4 5 
Grouping 5 3 
Grouping 6 6 
Full Year 9 
Fixed Effects 


Notes: Colors are used to indicate year groupings for fixed effects. P indicates pre-recession group. D indicates during 
recession group. A indicates after recession group. X indicates individual year fixed effect. The rows correspond to the 
different estimation results in columns 2-9 of tables 6A-C. 
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Table 6A: Group IV — NAEP 


dd) (2) (3) (4) (5) (6) (7) (8) (9) (10) 
Including Washington D.C. Excluding Washington D.C. 
oe JWx 10" and 90" = 15and 85 = 20and 80" — 25" and 75" 10% and 90" = 15and85" © 20and 80" = — 25" and 75" 
Result Replication Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles 
Per-Pupil Spending (thousands) 0.0385*** 0.0472** 0.0389 0.0266 -0.0037 -0.0197 0.0197 -0.0064 -0.0190 -0.0363 
(0.0110) (0.0172) (0.0218) (0.0226) (0.0221) (0.0327) (0.0215) (0.0257) (0.0226) (0.0369) 
Pre-Recession State Trends x x x x x x x x x x 
Year Fixed Effects x x x x x x x x x x 
Bartik Economic Controls x x x x x x x x x x 
States 51 51 51 51 51 51 50 50 50 50 
State- Year Observations 459 459 459 459 459 459 450 450 450 450 
Notes: The Original JWX Result is from Table 3 column 4 of JWX (2020a). *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent 
level. 
Table 6B: Group IV — College-Going Rate 
dd) (2) (3) (4) (5) (6) (7) (8) (9) (10) 
Including Washington D.C. Excluding Washington D.C. 
ies JWx 10" and 90" 15"and 85"  20and 80" 25" and 75" 10°and90" =15"and 85"  20"and 80" 25" and 75% 
Result Replication Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles 
Per-Pupil Spending (thousands) 0.0124*** 0.0203** 0.0212 0.0179 0.0147 0.0148 0.0089 0.0132 0.0086 0.0080 
(0.00387) (0.0066) (0.0122) (0.0122) (0.0112) (0.0137) (0.0132) (0.0122) (0.0114) (0.0132) 
Pre-Recession State Trends x x x x x x x x x x 
Year Fixed Effects x x x x x x x x x x 
Bartik Economic Controls x x x x x x x x x x 
States 51 51 51 51 51 51 50 50 50 50 
State- Year Observations 459 459 459 459 459 459 450 450 450 450 
Notes: The Original JWX Result is from Table 3 column 8 of JWX (2020a). *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent 
level. 
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Table 6C: Group IV — High School Graduation Rate 


qd) (2) (3) (4) (5) (6) 2) (8) (9) (10) 
Including Washington D.C. Excluding Washington D.C. 
ae teas JWx 10" and 90" 15" and 85" = 20" and 80" =—-.25" and 75" 10°and90" = =15"and 85"  20"and 80" = 25" and 75% 
Result Replication Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles Percentiles 
Per-Pupil Spending (thousands) - 0.0132 0.0424 0.0323 0.0144 0.0081 0.0335 0.0243 0.0093 0.0008 
- (0.0094) (0.0345) (0.0287) (0.0220) (0.0258) (0.0377) (0.0314) (0.0216) (0.0239) 
Pre-Recession State Trends - x x xX xX x x x x x 
Year Fixed Effects - xX xX xX xX x x x x x 
Bartik Economic Controls - x x xX x Xx x x x x 
States - 51 51 51 51 51 50 50 50 50 
State- Year Observations - 867 867 867 867 867 850 850 850 850 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 


Table 7A: Linear IV — NAEP 


qd) (2) (3) (4) (5) (6) 2) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWxX JWx JWx Full Year 
Results Grouping Grouping __Grouping2 — Grouping3_—_Grouping4 — Grouping5 __Grouping6 __ Fixed Effects 
Per-Pupil Spending (thousands) 0.0316*** 0.0304*** 0.0192*** -0.0082 -0.0340*** -0.0380* -0.0088 -0.0215 0.0000 
(0.0127) (0.0095) (0.0060) (0.0337) (0.0129) (0.0211) (0.0096) (0.0200) (0.0337) 
F-Statistic 41.61 33.77 31.88 1.16 17.55 7.92 16.78 6.86 1.68 
Pre-Recession State Trends x x x xX x x Xx xX x 
Bartik Economic Controls x x x x x x x x Xx 
States 51 51 50 50 50 50 50 50 50 
State- Year Observations 459 459 450 450 450 450 450 450 450 


Notes: The Original JWX Result is from Table A23 column 4 of JWX (2020a). *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 
percent level. 
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Table 7B: Linear IV — College-Going Rate 


dd) (2) (3) (4) (5) (6) 2) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWx JWx Jwx Full Year 
Results Grouping Grouping __Grouping2_ — Grouping3_ Grouping 4 — Grouping5 __Grouping6 __ Fixed Effects 
Per-Pupil Spending (thousands) 0.0240***  0.0146*** 0.0145*** — 0.0386*** 0.0227*** 0.0136*** 0.0137*** 0.0085*** 0.0145 
(0.00621) (0.0020) (0.0022) (0.0119) (0.0044) (0.0044) (0.0022) (0.0023) (0.0131) 
F-Statistic 34.13 36.22 32.42 6.11 18.05 11.10 34.12 27.11 1.59 
Pre-Recession State Trends xX x xX xX x x x x x 
Bartik Economic Controls xX x xX xX x x x x x 
States 51 51 50 50 50 50 50 50 50 
State- Year Observations 459 459 450 450 450 450 450 450 450 


Notes: The Original JWX Result is from Table A23 column 8 of JWX (2020a). *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 


percent level. 


Table 7C: Linear IV — High School Graduation Rate 


qd) (2) (3) (4) (5) (6) 2) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWx JWx JWx Full Year 
Results Grouping Grouping __Grouping2 — Grouping3_ Grouping 4 — Grouping5 __Grouping6__ Fixed Effects 
Per-Pupil Spending (thousands) - -0.0054 -0.0078* 0.0134* -0.0007 0.0055 -0.0075* -0.0040 -0.0102 
- (0.0045) (0.0046) (0.0070) (0.0069) (0.0072) (0.0046) (0.0046) (0.0221) 
F-Statistic 47.09 37.53 12.68 18.19 15.62 39.48 35.48 1.84 
Pre-Recession State Trends - x xX xX x x x x x 
Bartik Economic Controls - x xX xX x x x x x 
States - 51 50 50 50 50 50 50 50 
State- Year Observations - 867 850 850 850 850 850 850 850 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 
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Table 8A: Linear IV using Property Tax Share - NAEP 


dd) (2) (3) (4) (5) (6) 2) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWx JWx JWx Full Year 
Results Grouping Grouping __Grouping2 Grouping 3_—_ Grouping 4 — Grouping5 __Grouping6 __ Fixed Effects 
Per-Pupil Spending (thousands) - 0.0306*** 0.0197** -0.0084 -0.0453** -0.0294 -0.0239 -0.0224 -0.0060 
- (0.0100) (0.0084) (0.0207) (0.0181) (0.0226) (0.0147) (0.0230) (0.0315) 
F-Statistic 18.95 49.44 5.42 17.06 14.67 17.19 11.41 1.46 
Pre-Recession State Trends - x x x x x x xX x 
Bartik Economic Controls - x x x x x x x x 
States - 51 50 50 50 50 50 50 50 
State- Year Observations - 459 450 450 450 450 450 450 450 
Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 
Table 8B: Linear IV using Property Tax Share — College-Going Rate 
qd) (2) (3) (4) (5) (6) 2) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWx JWX JWx Full Year 
Results Grouping Grouping Grouping 2_—_Grouping3__ Grouping 4 Grouping 5 Grouping 6__ Fixed Effects 
Per-Pupil Spending (thousands) - 0.0116*** 0.0138*** 0.0085 0.0171** 0.0152** 0.0116*** 0.0094*** 0.0095 
- (0.0026) (0.0033) (0.0061) (0.0068) (0.0066) (0.0029) (0.0031) (0.0127) 
F-Statistic 18.04 53.81 17.64 25.12 22.18 63.63 60.89 1.30 
Pre-Recession State Trends - x x x x x Xx x x 
Bartik Economic Controls - x x x x x x xX x 
States - 51 50 50 50 50 50 50 50 
State- Year Observations - 459 450 450 450 450 450 450 450 


Notes: *** Significant at the 1 percent level. 


** Significant at the 5 percent level. * Significant at the 10 percent level. 
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Table 8C: Linear IV using Property Tax Share — High School Graduation Rate 


dd) (2) (3) (4) (5) (6) ) (8) (9) 
Including 
Washington D.C. Excluding Washington D.C. 
Original 
JWx JWx JWx Full Year 
Results Grouping Grouping __Grouping2_ — Grouping3_ Grouping 4 — Grouping5 __Grouping6__ Fixed Effects 
Per-Pupil Spending (thousands) - 0.0005 -0.0002 0.0150 0.0285** 0.0211* -0.0001 0.0018 -0.0006 
- (0.0032) (0.0037) (0.0108) (0.0115) (0.0116) (0.0037) (0.0042) (0.0205) 
F-Statistic 24.61 45.42 16.35 11.60 13.29 45.02 43.78 2.01 
Pre-Recession State Trends - x xX xX x x x x x 
Bartik Economic Controls - x Xx xX x x x x x 
States - 51 50 50 50 50 50 50 50 
State- Year Observations - 867 850 850 850 850 850 850 850 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 


Table 9: Replace Instrument with State Share of Higher Education Revenue 


dd) (2) (3) (4) (5) (6) 2) (8) 
NAEP Scores College-Going Rate 
Original JWxX No Year JWx Original JWxX No Year JwWx 
JWX Grouping Fixed Grouping JWxX Grouping Fixed Grouping 
Result from Table 6A Effects Result from Table 6B Effects 
Per-Pupil Spending (thousands)  0.0316*** 0.0304*** 0.0325*** — 0.0293*** 0.0240*** 0.0146*** 0.0196***  0.0116*** 
(0.0127) (0.0095) (0.0123) (0.0093) (0.00621) (0.0020) (0.0040) (0.0024) 
Pre-Recession State Trends x Xx x Xx x x x x 
Year-Group Fixed Effects x xX x xX x x x xX 
Bartik Economic Controls x Xx x xX x x x x 
States 51 51 51 51 51 51 51 51 
State- Year Observations 459 459 459 459 459 459 459 459 


Notes: The Original JWX Result for NAEP scores is from Table A23 column 4 of JWX (2020a). The Original JWX Result for college-going rate is from Table A23 column 8 of 
JWX (2020a).*** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 
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Table 10A: Predict State Appropriations Per FTE for Higher Education 


dd) (2) (3) (4) (5) (6) (7) (8) 
No Year JWxX Full Year 
Fixed Effects | Grouping Grouping 2. Grouping 3. Grouping 4 ~~ Grouping5 ~~ Grouping6 __ Fixed Effects 
Per-Pupil Spending (thousands) 530.16*** 399.46*** 225.30 567.66*** 332.17* 377.20*** 255.56** 701.59** 
(136.71) (109.28) (254.78) (164.12) (181.94) (111.81) (119.91) (326.25) 
Pre-Recession State Trends x x x x x x x x 
Bartik Economic Controls x x x x x x x x 
States 50 50 50 50 50 50 50 50 
State-Year Observations 850 850 850 850 850 850 850 850 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 


Table 10B: Predict Tuition and Fees Per FTE for Higher Education 


dd) (2) (3) (4) (5) (6) (7) (8) 
No Year JWxX Full Year 
Fixed Effects Grouping Grouping2 Grouping 3__Grouping4_—_Grouping5 Grouping 6 _ Fixed Effects 
Per-Pupil Spending (thousands) 336.35*** 257.58*** 476.16** 227.28** 285.38** 258.02*** 285.75*** 199.13 
(104.69) (71.29) (236.66) (112.03) (144.14) (75.31) (86.84) (281.88) 
Pre-Recession State Trends x x x x x x x x 
Bartik Economic Controls x x x x x x x x 
States 50 50 50 50 50 50 50 50 
State-Year Observations 850 850 850 850 850 850 850 850 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 


Table 11: Linear IV for College-Going using Log State Appropriations Per FTE and Log Tuition Per FTE 


d) (2) (3) (4) (5) (6) (7) (8) 
Higher Education Share JWX K-12 State Share 
State Appropriations Tuition State Appropriations Tuition 
No Year Fixed No Year Fixed JWxX No Year Fixed JWx No Year Fixed Jwx 
Effects JWX Grouping Effects Grouping Effects Grouping Effects Grouping 
Ln (State Approps or Tuition) 0.1831*** 0.1185** 0.1884*** 0.1321*** 0.2441 *** 0.1883*** 0.2269*** 0.1596*** 
(0.0616) (0.0543) (0.0587) (0.0506) (0.0589) (0.0582) (0.0630) (0.0487) 
Pre-Recession State Trends xX x x xX x x xX x 
Bartik Economic Controls xX x x xX xX x x x 
States 50 50 50 50 50 50 50 50 
State- Year Observations 450 450 450 450 450 450 450 450 


Notes: *** Significant at the 1 percent level. ** Significant at the 5 percent level. * Significant at the 10 percent level. 
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